Papers with multilingual sequence-to-sequence pretraining
PARADISE: Exploiting Parallel Data for Multilingual Sequence-to-Sequence Pretraining (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing methods for multilingual sequence-to-sequence pretraining rely on monolingual corpora and do not use strong cross-lingual signal contained in parallel data. |
| Approach: | They propose a method that replaces monolingual words with a bilingual dictionary and predicts the reference translation according to a parallel corpus instead of recovering the original sequence. |
| Outcome: | The proposed method improves machine translation and cross-lingual natural language inference by 2.0 BLEU points and 6.7 accuracy points over existing methods at a fraction of their computational cost. |